Interpretable Neural Network Research Articles

Evolutionary Perspectives on Neural Network Generations: A Critical Examination of Models and Design Strategies

Abstract: In the last few years, Neural Networks have become more common in different areas due to their ability to learn intricate patterns and provide precise predictions. Nonetheless, creating an efficient neural network model is a difficult task that demands careful thought of multiple factors, such as architecture, optimization method, and regularization technique. This paper aims to comprehensively overview the state-of-the-art artificial neural network (ANN) generation and highlight key challenges and opportunities in machine learning applications. It provides a critical analysis of current neural network model design methodologies, focusing on the strengths and weaknesses of different approaches. Also, it explores the use of different deep neural networks (DNN) in image recognition, natural language processing, and time series analysis. In addition, the text explores the advantages of selecting optimal values for various components of an Artificial Neural Network (ANN). These components include the number of input/output layers, the number of hidden layers, the type of activation function used, the number of epochs, and the model type selection. Setting these components to their ideal values can help enhance the model's overall performance and generalization. Furthermore, it identifies some common pitfalls and limitations of existing design methodologies, such as overfitting, lack of interpretability, and computational complexity. Finally, it proposes some directions for future research, such as developing more efficient and interpretable neural network architectures, improving the scalability of training algorithms, and exploring the potential of new paradigms, such as Spiking Neural Networks, quantum neural networks, and neuromorphic computing.

CTV-Net: Complex-Valued TV-Driven Network With Nested Topology for 3-D SAR Imaging.

The regularization-based approaches offer promise in improving synthetic aperture radar (SAR) imaging quality while reducing system complexity. However, the widely applied l1 regularization model is hindered by their hypothesis of inherent sparsity, causing unreal estimations of surface-like targets. Inspired by the edge-preserving property of total variation (TV), we propose a new complex-valued TV (CTV)-driven interpretable neural network with nested topology, i.e., CTV-Net, for 3-D SAR imaging. In our scheme, based on the 2-D holography imaging operator, the CTV-driven optimization model is constructed to pursue precise estimations in weakly sparse scenarios. Subsequently, a nested algorithmic framework, i.e., complex-valued TV-driven fast iterative shrinkage thresholding (CTV-FIST), is derived from the theory of proximal gradient descent (PGD) and FIST algorithm, theoretically supporting the design of CTV-Net. In CTV-Net, the trainable weights are layer-varied and functionally relevant to the hyperparameters of CTV-FIST, which aims to constrain the algorithmic parameters to update in a well-conditioned tendency. All weights are learned by end-to-end training based on a two-term cost function, which bounds the measurement fidelity and TV norm simultaneously. Under the guidance of the SAR signal model, a reasonably sized training set is generated, by randomly selecting reference images from the MNIST set and consequently synthesizing complex-valued label signals. Finally, the methodology is validated, numerically and visually, by extensive SAR simulations and real-measured experiments, and the results demonstrate the viability and efficiency of the proposed CTV-Net in the cases of recovering 3-D SAR images from incomplete echoes.

GINN-LP: A Growing Interpretable Neural Network for Discovering Multivariate Laurent Polynomial Equations

Traditional machine learning is generally treated as a black-box optimization problem and does not typically produce interpretable functions that connect inputs and outputs. However, the ability to discover such interpretable functions is desirable. In this work, we propose GINN-LP, an interpretable neural network to discover the form and coefficients of the underlying equation of a dataset, when the equation is assumed to take the form of a multivariate Laurent Polynomial. This is facilitated by a new type of interpretable neural network block, named the “power-term approximator block”, consisting of logarithmic and exponential activation functions. GINN-LP is end-to-end differentiable, making it possible to use backpropagation for training. We propose a neural network growth strategy that will enable finding the suitable number of terms in the Laurent polynomial that represents the data, along with sparsity regularization to promote the discovery of concise equations. To the best of our knowledge, this is the first model that can discover arbitrary multivariate Laurent polynomial terms without any prior information on the order. Our approach is first evaluated on a subset of data used in SRBench, a benchmark for symbolic regression. We first show that GINN-LP outperforms the state-of-the-art symbolic regression methods on datasets generated using 48 real-world equations in the form of multivariate Laurent polynomials. Next, we propose an ensemble method that combines our method with a high-performing symbolic regression method, enabling us to discover non-Laurent polynomial equations. We achieve state-of-the-art results in equation discovery, showing an absolute improvement of 7.1% over the best contender, by applying this ensemble method to 113 datasets within SRBench with known ground-truth equations.

Abstract 912: CheckpointPx, an interpretable radiology AI tool, predicts checkpoint blockade benefit independent of PDL1 status in non-small cell lung cancers (NSCLC): A multi-institutional validation study

Abstract Background: Immune Checkpoint Inhibitors (ICIs) are a cornerstone in the treatment of cancers such as Non-Small Cell Lung Cancer (NSCLC). Despite the standard use of biomarkers such as PD-L1 expression for selecting ICI regimens, recent studies suggest missed opportunities for benefit in PD-L1 low/negative patients. Therefore, there is an urgent need for new biomarker strategies capable of guiding the use of ICIs in NSCLC. We introduce and validate an AI-powered radiology tool for ICI patient selection that provides interpretable predictions by quantifying aspects of tumor biology on baseline CT scans directly associated with ICI efficacy, such as tumor heterogeneity and twistedness of the tumor-associated vasculature. Methods: We analyzed CT scans of 439 patients treated with ICI at 4 institutions (D1-D4) using the Picture Health Px platform. Experienced radiologists and physicians across institutions delineated target lesions on baseline CT scans. A deep learning model was used to segment pulmonary vessels. CheckpointPx v1.11, an interpretable neural network incorporating quantitative features extracted from within the tumor and its vasculature, was trained to predict ICI response (defined as disease control per RECIST best overall response) on n=247 patients (D1-D3). It was assessed with respect to response and progression free-survival (PFS) on 192 patients (D2-D4), with D4 (n=105) being an external validation set. CheckpointPx was also evaluated within PDL1 subsets among testing set patients where available (n=138). Results: The cohort was predominantly late stage (Stage 3/4) (>85%) and was mixed with respect to ICI line of treatment (1st-4th). 122 of 247 (49%) and 119 of 192 (62%) were responders in training and test cohorts respectively. CheckpointPx included 16 features, such as entropy-based heterogeneity and quantitative vessel tortuosity (twistedness). The model predicted response to ICI with an AUC=0.65 on the test set. The CheckpointPx High Risk group significantly stratified patients by PFS (HR=1.67 [1.22-2.28], p=0.001). This separation remained significant within the subset of PDL1-negative (HR=2.71 [1.35-5.44], p=0.005) and PDL1-positive patients (HR=2.05 [1.22-3.46], p=0.007). Conclusions: From baseline radiology, CheckpointPx was shown to strongly predict ICI outcomes across multiple institutions, NSCLC stages, and lines of ICI. Furthermore, the tool stratified patients by ICI benefit within both PDL1-positive and negative subsets - suggesting the potential of CheckpointPx to address critical gaps in the NSCLC biomarker landscape. CheckpointPx is driven by interpretable imaging biomarkers, and thus its predictions are tied to specific and quantifiable phenotypic attributes of the tumor and its microenvironment. Citation Format: Amogh Hiremath, Seyoung Lee, Jeeyeon Lee, Peter Haseok Kim, Kai Zhang, Salie Lee, Monica Yadav, Maria Jose Aguilera Chuchuca, Taegyu Um, Myungwoo Nam, Liam Il-Young Chung, Hye Sung Kim, Jisang Yu, Trie Arni Djunadi, Leeseul Kim, Youjin Oh, Sungmi Yoon, Zunairah Shah, Yuchan Kim, Ilene Hong, Grace Kang, Jessica Jang, Amy Cho, Soowon Lee, Cecilia Nam, Timothy Hong, Yury S. Velichko, Amit Gupta, Vamsidhar Velcheti, Anant Madabhushi, Nathaniel Braman, Young Kwang Chae. CheckpointPx, an interpretable radiology AI tool, predicts checkpoint blockade benefit independent of PDL1 status in non-small cell lung cancers (NSCLC): A multi-institutional validation study [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 912.

Joint intra and inter-channel nonlinear compensation scheme based on improved learned digital back propagation for WDM systems.

In this paper, we improve the learned digital back propagation (LDBP) and propose a novel joint intra and inter-channel nonlinearity compensation scheme for polarization division multiplexing wavelength-division multiplexed (PDM-WDM) systems. From the perspective of interpretable neural network, the scheme realizes the alternating compensation of chromatic dispersion (CD) and nonlinearity based on physical models. The chromatic dispersion compensation (CDC) adopts one-dimensional convolution operation in the time domain. Moreover, the pulse-broadening effect is introduced into the overlap-and-save method. For nonlinear compensation, the improved joint model is applied, and the impact of the intra-channel pulse broadening and the walk-off effect between different channels caused by CD on the nonlinear effect is considered. To validate the effectiveness of the proposed scheme, we construct an 11-channel simulation system of 36 GBaud PDM uniform 16 quadrature amplitude modulation (PDM-16QAM) 1600 km and 64 GBaud PDM-64QAM 400 km, as well as a 5-channel experimental system of 28 GBaud PDM-16QAM 806.4 km. The simulation results show that the performance of PDM-16QAM with 0.5 steps per span and PDM-64QAM with 2 steps per span improve the Q-factor by approximately 0.75 dB and 0.54 dB at the optimal launch power, compared with the linear compensation scheme. The transmission performance of PDM-16QAM is higher than that of digital back propagation with 5 steps per span (DBP-5StPS), and the complexity is only 31.36% of that of DBP-5StPS. The performance of PDM-64QAM is higher than that of DBP-10StPS, with a complexity of 62.72%. The experimental results show that the performance of PDM-16QAM with 0.5 steps per span is improved with 0.86 dB Q-factor improvement compared with the linear compensation scheme at the optimal launch power, and the performance of the proposed scheme is higher than that of DBP-5StPS with a complexity of only 23.68%.

Incorporating metabolic activity, taxonomy and community structure to improve microbiome-based predictive models for host phenotype prediction

ABSTRACT We developed MicroKPNN, a prior-knowledge guided interpretable neural network for microbiome-based human host phenotype prediction. The prior knowledge used in MicroKPNN includes the metabolic activities of different bacterial species, phylogenetic relationships, and bacterial community structure, all in a shallow neural network. Application of MicroKPNN to seven gut microbiome datasets (involving five different human diseases including inflammatory bowel disease, type 2 diabetes, liver cirrhosis, colorectal cancer, and obesity) shows that incorporation of the prior knowledge helped improve the microbiome-based host phenotype prediction. MicroKPNN outperformed fully connected neural network-based approaches in all seven cases, with the most improvement of accuracy in the prediction of type 2 diabetes. MicroKPNN outperformed a recently developed deep-learning based approach DeepMicro, which selects the best combination of autoencoder and machine learning approach to make predictions, in all of the seven cases. Importantly, we showed that MicroKPNN provides a way for interpretation of the predictive models. Using importance scores estimated for the hidden nodes, MicroKPNN could provide explanations for prior research findings by highlighting the roles of specific microbiome components in phenotype predictions. In addition, it may suggest potential future research directions for studying the impacts of microbiome on host health and diseases. MicroKPNN is publicly available at https://github.com/mgtools/MicroKPNN.

A hybrid neural network approach for classifying diabetic retinopathy subtypes.

Diabetic retinopathy is a prevalent complication among diabetic patients that, if not predicted and treated promptly, can lead to blindness. This paper proposes a method for accurately and swiftly predicting the degree of diabetic retinopathy using a hybrid neural network model. Timely prediction of diabetic retinopathy is crucial in preventing blindness associated with this condition. This study aims to enhance the prediction accuracy of diabetic retinopathy by utilizing the hybrid neural network model EfficientNet and Swin Transformer. The specific methodology includes: (1) combining local and global features to accurately capture lesion characteristics by leveraging the strengths of both Swin Transformer and EfficientNet models; (2) improving prediction accuracy through a comprehensive analysis of the model's training details and applying data augmentation techniques such as Gaussian blur to enhance the hybrid model's performance; (3) validating the effectiveness and utility of the proposed hybrid model for diabetic retinopathy detection through extensive experimental evaluations and comparisons with other deep learning models. The hybrid model was trained and tested on the large-scale real-world diabetic retinopathy detection dataset APTOS 2019 Blindness Detection. The experimental results show that the hybrid model in this paper achieves the best results in all metrics, including sensitivity of 0.95, specificity of 0.98, accuracy of 0.97, and AUC of 0.97. The performance of the model is significantly improved compared to the mainstream methods currently employed. In addition, the model provides interpretable neural network details through class activation maps, which enables the visualization of diabetic retinopathy. This feature helps physicians to make more accurate diagnosis and treatment decisions. The model proposed in this paper shows higher accuracy in detecting and diagnosing diabetic retinopathy, which is crucial for the treatment and rehabilitation of diabetic patients. The hybrid neural network model based on EfficientNet and Swin Transformer significantly contributes to the prediction of diabetic retinopathy. By combining local and global features, the model achieves improved prediction accuracy. The validity and utility of the model are verified through experimental evaluations. This research provides robust support for the early diagnosis and treatment of diabetic patients.

Synchronization-Inspired Interpretable Neural Networks.

Synchronization is a ubiquitous phenomenon in nature that enables the orderly presentation of information. In the human brain, for instance, functional modules such as the visual, motor, and language cortices form through neuronal synchronization. Inspired by biological brains and previous neuroscience studies, we propose an interpretable neural network incorporating a synchronization mechanism. The basic idea is to constrain each neuron, such as a convolution filter, to capture a single semantic pattern while synchronizing similar neurons to facilitate the formation of interpretable functional modules. Specifically, we regularize the activation map of a neuron to surround its focus position of the activated pattern in a sample. Moreover, neurons locally interact with each other, and similar ones are synchronized together during the training phase adaptively. Such local aggregation preserves the globally distributed representation nature of the neural network model, enabling a reasonably interpretable representation. To analyze the neuron interpretability comprehensively, we introduce a series of novel evaluation metrics from multiple aspects. Qualitative and quantitative experiments demonstrate that the proposed method outperforms many state-of-the-art algorithms in terms of interpretability. The resulting synchronized functional modules show module consistency across data and semantic specificity within modules.

A Hybrid Model for Spatiotemporal Air Quality Prediction Based on Interpretable Neural Networks and a Graph Neural Network

To effectively address air pollution and enhance air quality, governments must be able to predict the air quality index with high accuracy and reliability. However, air quality prediction is subject to ambiguity and instability because of the atmosphere’s fluidity, making it challenging to identify the temporal and spatial correlations using a single model. Therefore, a new hybrid model is proposed based on an interpretable neural network and a graph neural network (INNGNN), which simulates the temporal and spatial dependence of air quality and achieves accurate multi-step air quality prediction. A time series is first interpreted using interpretable neural networks (INN) to extract the potentially important aspects that are easily overlooked in the data; second, a self-attention mechanism catches the local and global dependencies and associations in the time series. Lastly, a city map is created using a graph neural network (GNN) to determine the relationships between cities in order to extract the spatially dependent features. In the experimental evaluation, the results show that the INNGNN model performs better than comparable algorithms. Therefore, it is confirmed that the INNGNN model can effectively capture the temporal and spatial relationships and better predict air quality.

Optimizing Seasonal‐To‐Decadal Analog Forecasts With a Learned Spatially‐Weighted Mask

AbstractSeasonal‐to‐decadal climate prediction is crucial for decision‐making in a number of industries, but forecasts on these timescales have limited skill. Here, we develop a data‐driven method for selecting optimal analogs for seasonal‐to‐decadal analog forecasting. Using an interpretable neural network, we learn a spatially‐weighted mask that quantifies how important each grid point is for determining whether two climate states will evolve similarly. We show that analogs selected using this weighted mask provide more skillful forecasts than analogs that are selected using traditional spatially‐uniform methods. This method is tested on two prediction problems using the Max Planck Institute for Meteorology Grand Ensemble: multi‐year prediction of North Atlantic sea surface temperatures, and seasonal prediction of El Niño Southern Oscillation. This work demonstrates a methodical approach to selecting analogs that may be useful for improving seasonal‐to‐decadal forecasts and understanding their sources of skill.

Interpretable real-time monitoring of pipeline weld crack leakage based on wavelet multi-kernel network

The deep learning technology used for pipeline weld crack leakage monitoring lacks physical interpretability, which makes it difficult to provide theoretical support to decision-makers, reducing the practicality and applicability of the technology. To solve this problem, a deep learning framework that combines interpretability and feature fusion is proposed to solve the real-time monitoring problem of pipeline leaks. First, three different wavelet basis functions are used to analyze the leakage acoustic emission signal, and the resulting time-frequency features exhibit correlation and complementarity. Based on this, an interpretable neural network with different wavelet convolution kernels is designed to extract abstract feature details of wideband acoustic emission signals through multi-level dynamic receptive fields. Then, a feature fusion module based on channel importance weighting was designed to optimize the learning process of the network by highlighting the contribution of different channels. Finally, the effectiveness and superiority of the proposed method were verified through pipeline leakage acoustic emission monitoring experiments. The results show that the proposed method can effectively extract distinguishing features of leak acoustic emission signals, and the recognition accuracy of different leakage states can reach 98.32%, which is significantly higher than that of typical deep learning methods. In addition, feature map visualization proves that the proposed method has physical interpretability in abstract feature extraction, which will provide a reference basis for correct decision-making by industrial technicians.

Interpretable unsupervised learning enables accurate clustering with high-throughput imaging flow cytometry

A primary challenge of high-throughput imaging flow cytometry (IFC) is to analyze the vast amount of imaging data, especially in applications where ground truth labels are unavailable or hard to obtain. We present an unsupervised deep embedding algorithm, the Deep Convolutional Autoencoder-based Clustering (DCAEC) model, to cluster label-free IFC images without any prior knowledge of input labels. The DCAEC model first encodes the input images into the latent representations and then clusters based on the latent representations. Using the DCAEC model, we achieve a balanced accuracy of 91.9% for human white blood cell (WBC) clustering and 97.9% for WBC/leukemia clustering using the 3D IFC images and 3D DCAEC model. Above all, although no human recognizable features can separate the clusters of cells with protein localization, we demonstrate the fused DCAEC model can achieve a cluster balanced accuracy of 85.3% from the label-free 2D transmission and 3D side scattering images. To reveal how the neural network recognizes features beyond human ability, we use the gradient-weighted class activation mapping method to discover the cluster-specific visual patterns automatically. Evaluation results show that the automatically identified salient image regions have strong cluster-specific visual patterns for different clusters, which we believe is a stride for the interpretable neural network for cell analysis with high-throughput IFCs.

TFN: An interpretable neural network with time-frequency transform embedded for intelligent fault diagnosis

Convolutional neural network (CNN) is widely used in fault diagnosis of mechanical systems due to its powerful feature extraction and classification capabilities. However, the CNN is a typical black-box model, and the mechanism of CNN’s decision-making is not clear, which limits its application in high-reliability-required fault diagnosis scenarios. To tackle this issue, we propose a novel interpretable neural network termed as time-frequency network (TFN), where the physically meaningful time-frequency transform (TFT) method is embedded into the traditional convolutional layer as a trainable preprocessing layer. This preprocessing layer named as time-frequency convolutional (TFconv) layer, is constrained by a well-designed kernel function to extract fault-related time-frequency information. It not only improves the diagnostic performance but also reveals the logical foundation of the CNN prediction in a frequency domain view. Different TFT methods correspond to different kernel functions of the TFconv layer. In this study, three typical TFT methods are considered to formulate the TFNs and their diagnostic effectiveness and interpretability are proved through three mechanical fault diagnosis experiments. Experimental results also show that the proposed TFconv layer has outstanding advantages in convergence speed and few-shot scenarios, and can be easily generalized to other CNNs with different depths to improve their diagnostic performances. The code of TFN is available on https://github.com/ChenQian0618/TFN.

A health status estimation method based on interpretable neural network observer for HVs

Estimating the health status is a crucial step in learning about the health of hypersonic vehicles beforehand. The estimation results can be used to detect abnormal states and provide data reference for fault diagnosis. However, certain conventional neural network-based estimate techniques rely heavily on data and have limited model interpretability, which challenges the accuracy of the estimation results. This research aims to address the problems of data dependency and model interpretability in estimation models. In this study, a block interpretable neural network model with constraints on the trajectory and attitude equations is established. On the basis of the interpretable neural network model, two health status estimation methods are proposed: one that is unsupervised and the other that is supervised. Additionally, in the supervised health status estimate approach, an FC-LN-Mish structure is created to fit the relationship between the fault residual and the fault state parameters. The results indicate that the proposed estimation methods can fit the system mechanism relationship more accurately, improve the model interpretability, reduce data dependency, and ensure high estimation efficiency and precision. The FC-LN-Mish structure can reduce the missed detection rate and false detection rate to some extent, and perform better than other models under the low fault deviation degree. In conclusion, the interpretable neural network model-based observers accurately observe the health status parameters of rudders and RCS, reduce data dependence and data processing costs, and have better performance under high uncertainty interference. It provides effective method for online health estimation.

A depthwise separable CNN-based interpretable feature extraction network for automatic pathological voice detection

In recent years, deep learning methods in automatic pathological voice detection (APVD) have gained satisfying results. However, most deep learning methods in APVD cannot explain their performance. Interpretability is crucial in deep learning methods applied to the medical field. A lack of interpretability makes it hard for existing methods to give better generalization performance than meaningful feature-based methods in practical applications. This paper proposed an interpretable neural network architecture called the Interpretable Multi-band Feature Extraction Network (IMBFN) based on clear feature extraction logic and a comprehensive result judgment method to improve the effectiveness and generalization performance of APVD. An amplitude-trainable SincNet (AT-SincNet) filter bank was put forward in IMBFN and applied as the front-end frequency division network. In addition, IMBFN used a designed two-path one-dimensional depthwise separatable convolutional neural network (CNN)-based feature extractor to extract meaningful voice features. The classification results of each voice frame were used to judge whether the voice was pathological synthetically. Comparative experiments were conducted using data from the MEEI, SVD, and HUPA databases. The best improvement of accuracy, F1-score, and Matthews correlation coefficient (MCC) reached 0.1705, 0.1977, and 0.4463, respectively. Also, blind tests were carried out in participants from the First Affiliated Hospital of Soochow University, and an accuracy, F1-score, and MCC of 0.7594, 0.8491, and 0.2981, respectively, were obtained. Results demonstrated that IMBFN provided meaningful explanations, good APVD effect, and better generalization performance than existing methods.

DeepGAMI: deep biologically guided auxiliary learning for multimodal integration and imputation to improve genotype–phenotype prediction

BackgroundGenotypes are strongly associated with disease phenotypes, particularly in brain disorders. However, the molecular and cellular mechanisms behind this association remain elusive. With emerging multimodal data for these mechanisms, machine learning methods can be applied for phenotype prediction at different scales, but due to the black-box nature of machine learning, integrating these modalities and interpreting biological mechanisms can be challenging. Additionally, the partial availability of these multimodal data presents a challenge in developing these predictive models.MethodTo address these challenges, we developed DeepGAMI, an interpretable neural network model to improve genotype–phenotype prediction from multimodal data. DeepGAMI leverages functional genomic information, such as eQTLs and gene regulation, to guide neural network connections. Additionally, it includes an auxiliary learning layer for cross-modal imputation allowing the imputation of latent features of missing modalities and thus predicting phenotypes from a single modality. Finally, DeepGAMI uses integrated gradient to prioritize multimodal features for various phenotypes.ResultsWe applied DeepGAMI to several multimodal datasets including genotype and bulk and cell-type gene expression data in brain diseases, and gene expression and electrophysiology data of mouse neuronal cells. Using cross-validation and independent validation, DeepGAMI outperformed existing methods for classifying disease types, and cellular and clinical phenotypes, even using single modalities (e.g., AUC score of 0.79 for Schizophrenia and 0.73 for cognitive impairment in Alzheimer’s disease).ConclusionWe demonstrated that DeepGAMI improves phenotype prediction and prioritizes phenotypic features and networks in multiple multimodal datasets in complex brains and brain diseases. Also, it prioritized disease-associated variants, genes, and regulatory networks linked to different phenotypes, providing novel insights into the interpretation of gene regulatory mechanisms. DeepGAMI is open-source and available for general use.

An interpretable neural network for robustly determining the location and number of cluster centers

An interpretable neural network for robustly determining the location and number of cluster centers

Interpretable neural networks: principles and applications.

In recent years, with the rapid development of deep learning technology, great progress has been made in computer vision, image recognition, pattern recognition, and speech signal processing. However, due to the black-box nature of deep neural networks (DNNs), one cannot explain the parameters in the deep network and why it can perfectly perform the assigned tasks. The interpretability of neural networks has now become a research hotspot in the field of deep learning. It covers a wide range of topics in speech and text signal processing, image processing, differential equation solving, and other fields. There are subtle differences in the definition of interpretability in different fields. This paper divides interpretable neural network (INN) methods into the following two directions: model decomposition neural networks, and semantic INNs. The former mainly constructs an INN by converting the analytical model of a conventional method into different layers of neural networks and combining the interpretability of the conventional model-based method with the powerful learning capability of the neural network. This type of INNs is further classified into different subtypes depending on which type of models they are derived from, i.e., mathematical models, physical models, and other models. The second type is the interpretable network with visual semantic information for user understanding. Its basic idea is to use the visualization of the whole or partial network structure to assign semantic information to the network structure, which further includes convolutional layer output visualization, decision tree extraction, semantic graph, etc. This type of method mainly uses human visual logic to explain the structure of a black-box neural network. So it is a post-network-design method that tries to assign interpretability to a black-box network structure afterward, as opposed to the pre-network-design method of model-based INNs, which designs interpretable network structure beforehand. This paper reviews recent progress in these areas as well as various application scenarios of INNs and discusses existing problems and future development directions.

Clustering by sparse orthogonal NMF and interpretable neural network

Clustering by sparse orthogonal NMF and interpretable neural network

Predicting regional somatic mutation rates using DNA motifs.

How the locus-specificity of epigenetic modifications is regulated remains an unanswered question. A contributing mechanism is that epigenetic enzymes are recruited to specific loci by DNA binding factors recognizing particular sequence motifs (referred to as epi-motifs). Using these motifs to predict biological outputs depending on local epigenetic state such as somatic mutation rates would confirm their functionality. Here, we used DNA motifs including known TF motifs and epi-motifs as a surrogate of epigenetic signals to predict somatic mutation rates in 13 cancers at an average 23kbp resolution. We implemented an interpretable neural network model, called contextual regression, to successfully learn the universal relationship between mutations and DNA motifs, and uncovered motifs that are most impactful on the regional mutation rates such as TP53 and epi-motifs associated with H3K9me3. Furthermore, we identified genomic regions with significantly higher mutation rates than the expected values in each individual tumor and demonstrated that such cancer-related regions can accurately predict cancer types. Interestingly, we found that the same mutation signatures often have different contributions to cancer-related and cancer-independent regions, and we also identified the motifs with the most contribution to each mutation signature.